Learning Canonical Correlations of Paired Tensor Sets Via Tensor-to-Vector Projection

Canonical correlation analysis (CCA) is a useful technique for measuring relationship between two sets of vector data. For paired tensor data sets, we propose a multilinear CCA (MCCA) method. Unlike existing multilinear variations of CCA, MCCA extracts uncorrelated features under two architectures while maximizing paired correlations. Through a pair of tensor-to-vector projections, one architecture enforces zero-correlation within each set while the other enforces zero-correlation between different pairs of the two sets. We take a successive and iterative approach to solve the problem. Experiments on matching faces of different poses show that MCCA outperforms CCA and 2DCCA, while using much fewer features. In addition, the fusion of two architectures leads to performance improvement, indicating complementary information.